Probability and distributions

Random Process

In a Random Process we know what outcomes could happen, but we don´t know which particular outcome will happen.

Probability

Notation: P(A) = Probability of event A

Rule: 0 <= P(A) <= 1

Definition:

  • Frequentist Interpretation: The probability of an outcome is the proportion of times the outcome would occur if we observed the random process and infinite number of times.

  • Bayesian Interpretation: A bayesian interprets probability as a subjective degree of belief. Popular in the late twenty years.

Law of large numbers

Law of large numbers states that as more observations are collected (more random process), the proportion of occurrences with a particular outcome converges to the probability of that outcome

for example, we expect the Proportion of 5´s in a dice to set down to 1/6 with increasing number of rolls
Proportion of 5´s v.s number of rolls

Coin is memoryless: The probability of heads on the 11th toss is the same as the probability of heads in the 10th toss, or any previous tosses P(Head on the 11th toss) = P(Head on the 11th toss) = 0.5

Gambler´s fallacy/law of averages: Random processes are supposed to compensate for whatever happened in the past…. no, common misunderstanding of the law of large numbers

Disjoint Events + General Addition Rule

Disjoint Events

Disjoint Events (Mutually exclusive) cannot happen at the same time.
* A student can´t both fail and pass a class
* A singe card drawn from a deck cannot be an ace and a queen

In Venn diagram, where we represent each event by circles, if A and B are disjoint we end up with two circles that don´t touch each other
P

Non Disjoint Events

Non-Disjoint Events Can happen at the same time
* A student can get an A in Stats and A in Econ in the same semester

In Venn diagram,if A and B are non- disjoint we end up with two circles that join
Pf

Union of Disjoint Events

Union: Probability of one event or the other happening
Pf

Union of Non-Disjoint Events

We need to consider the join part of the two events in order to not double count and inflate our probability of a desire outcome
Pf

General addition rule

P(A or B) = P(A) + P(B) - P(A and B)

and when the events are disjoint (P(A and B) = 0)..

P(A or B) = P(A) + P(B)

Sample space

A Sample space is a collection of all possible outcomes of a trial

Pf

Probability distribution

A Probability distribution lists all possible outcomes in the sample space, and the probabilities with which thee occur

Pf

Rules * The events listed must be disjoint * Each probability must be between 0 and 1 * The probability must total 1 (the sum)

Complementary events

Complementary events are two mutually exclusive (disjoint) events whose probabilities add up to 1

Pf

Disjoint vs. complementary

Do the sum probabilities of two disjoint outcomes always add up to 1?

No there may be more than 2 outcomes in the sample space

Independence

Two processes are Independent if knowing the outcome of one provides no useful information about the outcome of the other

Checking for independence

Pf

Example

Pf

we use the expression Most likely dependent because we are dealing with sample data

If we observed difference between conditional probabilities (based in the sample) –> most likely dependent —> Hypothesis test to see if this difference is not due to chance

Rules with independent events

  • Product rule
Pf

Important Note random selection implies independence

—————————————————————————————————-

Observation: We can see a similarity between a dice roll and a sample

dice roll – we know the 6 options and their relative frequencies (each option repeat only one time) — if we roll the dice many times we get to the original relative frequencies.
The 6 options and their relative frequencies is like a population that we don´t know – if we take a sample of it we can get to the relative frequencies of the population
————————————————————————————————————————————————————————————

Disjoint vs. Independent

Disjointness is about events happening at the same time. While independence is about processes not affecting each other.

Pf

Note: Disjoint events with non zero probability are always dependent on each other.Because if we know that one happened, we know that the other one cannot happen

Pf

Conditional Probability

Marginal Probability

In a contingency table we look at the margins to calculate the marginal probabilities

Pf

Joint Probability

In a contingency table we look at the intersection of the events of interest to calculate the joint probability. Also, in a Venn diagram we can see the joint probability as the intersection between the circles.

Pf

Conditional Probability

In a contingency table, we fix a column or row and only use the information registered on it.

Pf

In a formal way, we have the Baye´s Theorem

Pf

With this formula now we can have an expression for the join probability of two events that are dependent

Pf

Independence and conditional probabilities

if two events are independent: P(A|B) = P(A) Conceptually: Giving B doesn´t tell us anything about A Mathematically:
Pf

Probability trees

See diapos

Bayesian Inference

With probability trees we can calculate a posterior probability

posterior probability: P(hypothesis|data): It tells us the probability of a hypothesis we set forth, given the data we just observed

depends on:

This is different than what we calculated at the end of the randomization test on gender discrimination - the probability of observed or more extreme data given the null hypothesis being true, i.e. P(data|hypothesis) also called a p-value

In the bayesian approach, we evaluate claims iteratively as we collect more data.

In other words, we update our prior with our posterior probability from the previous iteration.

Example

Pf
Pf
Pf